Efficient Caching Techniques for Server Network Acceleration

A majority of server applications such as web server, database, E-mail, storage, etc. process a lot of network data, making them very network I/O intensive. TCP/IP is the most commonly used network protocol by these applications. TCP/IP runs over Ethernet, which is the de-facto Local Area Network (LAN) protocol. Rapid growth of Internet-enabled applications has resulted in development of faster Ethernet network technologies (from 100 Mbps to 1 and 10 Gbps). This sudden jump in Ethernet speeds necessitates TCP/IP processing to scale proportionately so that the server applications can ultimately benefit. This turns out to be a big challenge for server platforms, given that the CPU and memory speeds improve at a slower rate (tracking Moore’s law). Work done in the past has shown that TCP/IP processing, especially the receive side, is very memory intensive and therefore limits achievable network throughput considerably. To understand this, we have done a thorough cache characterization of TCP/IP processing. Results from this work have pointed out that only a small fraction of TCP/IP data (TCB and hash nodes) exhibits temporal locality, while the majority (descriptors, protocol headers, and payload) exhibits no temporal locality and thus generates a lot of memory accesses. To address these issues, we use two caching techniques in this paper: (1) Cache Region Locking (CRL) with Automatic-Actions, and (2) Cache Region Prefetching (CRP). We describe these techniques and analyze their performance benefits. To measure performance benefits, we implemented these caching techniques in an execution-driven CPU simulator and applied these to a Free BSD TCP/IP stack. Our simulation results show that the proposed caching techniques improve TCP/IP performance up to 65% by reducing the memory stall time.